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Response to Arguments 
The indicated allowability of claims 25-26, 29-31, 33-34, and 37-39 are withdrawn in 
view of Applicant's submission of an information disclosure statement under 37 CFR 1.97(c) 
with the fee set forth in 37 CFR 1.1 7(p) on 8/6/2007 prompted the new ground(s) of rejection 
presented in this Office action. 
Claims 1-24, 27-28, 32, 35-36, 40-49 are cancelled. 

Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

Claims 25-26, 29-31, 33-34, and 37-39 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over H. S. Sawhney, et al. with title of "Video Flashlights - Real Time Rendering 
of Multiple Videos for Immersive Model Visualization", hereinafter Sawhney, see cited PTO- 
1449 dated 8/6/07. 

Claim 29. 

A method comprising: Sawhney teaches obtaining a three dimensional model of a three 
dimensional environment (e.g., under section 2 on page 158 in the third paragraph), the three 
dimensional model generated from range sensor (e.g., left col. line 5 on page 165), information 
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representing a height field for the three dimensional environment (e.g., a range finder device); 
identifying in real time a region in motion (e.g., the video sequence shown moving objects in fig. 
10) with respect to a background image (e.g., in fig. 10 see the background image) in real-time 
video imagery information from at least one image sensor having associated position and 
orientation information with respect to the three dimensional model (e.g., in fig. 10 see 3D 
change image), the background image comprising a single distribution background dynamically 
modeled from a time average of the real-time video imagery information (e.g., in fig. 10 
illustrates the background image and a video sequence, on page 167); placing a surface that 
corresponds to the moving region in the three dimensional model (e.g., in fig. 10 illustrates 
moving objects); projecting the real-time video imagery information onto the three dimensional 
model, including the surface, based on the position and orientation information (e.g., in fig. 10 
illustrates moving objects); and visualizing the three dimensional model with the projected real- 
time video imagery (e.g., in fig. 10 illustrates moving objects); wherein identifying a region in 
motion in real time comprises subtracting the background image from the real-time video 
imagery information (e.g., in fig. 10 illustrates on the right and left sides of the figure), 
identifying a foreground object in the subtracted real-time video imagery information, validating 
the foreground object by correlation matching between identified objects in neighboring image 
frames, and outputting the validated foreground object (e.g., in fig. 10 illustrates outputting the 
foreground object vs the background image at the right side of the figure); wherein identifying a 
foreground object comprises identifying the foreground object in the subtracted real-time video 
imagery information using a histogram-based threshold and a noise filter (e.g., on page 164 at the 
right col. teaches the absolute value at each point can then be computed, and the result can be 
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thresholded in order to highlight intensity or feature differences between the current video image 
and the reference image, on page 160 in fig. 3 clearly illustrates final rendered composite that has 
been filtered the video pixels, and Sawhney does not disclose "a noise filter" Examiner 
interpretation: the noise filter may be considered as a virtual camera view in fig. 3 (must be seen 
in color image) that shows just the video pixels); wherein identifying a region in motion in real 
time further comprises estimating the background image by modeling the background image as a 
temporal pixel average of five recent image frames in the real-time video imagery information 
(e.g., on page 164 at right col. teaches the reference background image needs to be constantly 
updated during the day to reflect changing ambient illumination. Alternatively only moving 
objects may be detected by comparing current image with an image taken a few seconds before. 
The 2D moving object detection can be done in real time). 

Sawhney does not explicitly specify (see the underlined sections) identifying a region in 
motion in real time by modeling the background image as a temporal pixel average of five recent 
image frames in the real time video imagery. However, Examiner takes an official notice for an 
average of five recent frames, because Sawhney on page 161 under "A pseudo-code for the 
rendering algorithm ..." teaches as one of the parameters that a user may be indicated a number 
for number of frames "frame number", and that number may be 5 image frames. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to substitute applicant 's described structure, material, or acts for that described in the Sawhney 's 
sysem/method, in order to take advantage of the Video Flashlight system that is the real-time 
rendering of multiple live videos overlaid precisely on a 3D model. 

Claim 25. 
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Sawhney teaches the method of claim 29, wherein the surface comprises a two dimensional 
surface (e.g., in fig. 5 illustrates 2D surfaces). 

Claim 26. 

The method of claim 25, wherein placing the two dimensional surface comprises: casting a ray 
(Examiner's note: casting rays are: cast and traced in groups based on some geometric 
constraints, and each ray is traced separately, so that every point (usually a pixel) on the display 
is traced by one ray.) from an optical center (e.g., in fig. 5 at the top shows a casting rays with 
respect to the bottom image that is considered every point on the display is traced by a ray, or in 
fig. 6 shows the concept of casting and ray tracing phenomena), corresponding to the real-time 
video imagery information, to a bottom point of the moving region in an image plane in the three 
dimensional model; and determining a position, an orientation and a size of the two dimensional 
surface based on the ray, a ground plane in the three dimensional model, and the moving region 
(e.g., in fig. 10 illustrates a 3D model, and a moving region). 

Claim 30. 

The method of claim 29, further comprising tracking the position (e.g., see moving objects 
represented as 3D masks. Only moving people or cars are detected) and orientation information 
of the at least one image sensor in the environment with respect to the three dimensional model 
in real-time (e.g., in fig. 10 illustrates a 3D model, see in 3Dchange image that detects moving 
people and cars). 

Claim 31. 
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The method of claim 30, wherein obtaining a three dimensional model of a three dimensional 
environment comprises generating the three dimensional model of the three dimensional 
environment (e.g., in fig. 8 illustrates a 3D model, see (d)). 

Claim 33. 

The system of claim 37, wherein the surface comprises a two dimensional surface (e.g., under 
section 2 at third paragraph teaches the 2D surface). 

Claim 34. 

Claim 34 is rejected with similar reason as set forth in claim 26, above. 
Claim 37. 

Claim 37 is rejected with similar reason as set forth in claim 29, above. 
Claim 38. 

Claim 38 is rejected with similar reason as set forth in claim 30, above. 
Claim 39. 

Claim 39 is rejected with similar reason as set forth in claim 31, above. 
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Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Javid A. Amini whose telephone number is 571-272-7654. The 
examiner can normally be reached on 8-4pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kee Tung can be reached on 571-272-7794. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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Examiner 
Art Unit 2628 
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